[Oneshot] Oneshot Refactor #1041
Conversation
|
👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review. |
Please update the PR desription showing what the updated entrypoint looks like as a result of this PR. |
|
@dsikka Thanks for looking at the pr early. Yes thats the next to do before the full review. One ready i will ping you! |
…gs and popualte model_args to avoid collision
horheynm
changed the title
| @@ -75,7 +75,6 @@ def test_oneshot_application(self): | |||
| model=self.model, | |||
| dataset=self.dataset, | |||
| output_dir=self.output, | |||
| overwrite_output_dir=True, | |||
There was a problem hiding this comment.
Only used in HF training args
https://github.com/huggingface/transformers/blob/main/src/transformers/training_args.py#L819
SUMMARY:
Edit oneshot pathway to use only the necessary code.
Problem:
Oneshot uses a pathway that is shared in other entrypoints, eg. train.
Oneshot doesnt train, so do not need use code that trains, as an example of redundant code. Remove other parts of the code and refactor
Design
Changes:
TEST PLAN:
Pass existing passing tests
Pass all finetune tests merging transformer main for HFQuantizer support
Verified new pathway has the same scales for the int8 w8a8 dynamic per token values for Meta-Llama-3-8B-Instruct